<!DOCTYPE HTML>
<html xmlns="http://www.w3.org/1999/xhtml" version="XHTML 1.2" xml:lang="en">
<head>
<title>Music Visualizer :: Derek Selander and Paul Craciunoiu</title>
</head>
<link type="text/css" rel="stylesheet" href="http://ajax.googleapis.com/ajax/libs/jqueryui/1.7.2/themes/smoothness/jquery-ui.css" />
<style>
h2 {
    text-align: center;
}
h3 {
    text-align: center;
}
table th
{
    text-align: center;
}
.done {
    text-decoration: line-through;
}
.critical {
    color: #e04444;
    font-weight: bold;
}
</style>

<body>
<h2 id="top">Music Visualizer</h2>
<h3>by Derek Selander and Paul Craciunoiu</h3>

<strong>Note: to view this page without tabs, you may choose to disable JavaScript.</strong><br/><br/>

<div id="tabs">
    <ul>
        <li><a href="#tabs-1">Overview</a></li>
        <li><a href="#tabs-2">Details</a></li>
        <li><a href="#tabs-3">Goals and Timeline</a></li>
        <li><a href="#tabs-4">References</a></li>
        <li><a href="#tabs-5">Media</a></li>
        <li><a href="#tabs-6">Checklist</a></li>
        <li><a href="#tabs-7">User's guide</a></li>
        <li><a href="#tabs-8">README</a></li>
    </ul>
    <div id="tabs-1">
        <h3 class="hide">Overview</h3>
        <a class="hide" href="#top">Back to top</a>
The objective of this project is to provide visual feedback based on input from reading an audio file. We will be using an audio library to read in data from the audio stream, and correlate that data with graphics output which, ideally, appears to be in tune with the music. We will have objects move around the screen and perform different types of actions depending on both the input and random variables. For example, we will have objects collide and deform, camera movements, and maybe new objects appear along with existing objects disappearing.
    </div><!-- #tabs-1 -->
    <div id="tabs-2">
        <h3 class="hide">Details</h3>
        <a class="hide" href="#top">Back to top</a>
<p>For details about running the program, see our <strong>User's guide section</strong>.</p>

<h4>Trajectories, Spheres, Environment</h4>

<p>This project consists of several spheres moving around trajectories determined by music. These trajectories are calculated using parametric curves (Hermite curves, more exactly). The primary reason we have chosen Hermite curves is because of their flexibility in trajectories. This comes from C0 continuity, which allows more flexible changes in trajectories, and thus allows trajectories to appear better synchronized with the music. The secondary reason is that generation of Hermite curves is very simple and thus allows for complex computation. During our first project iterations, our local laptops and computers successfully simulated up to 100 spheres at a framerate (per second) of above 30. (As a side note, my local configs are: Intel Core 2 Duo 2.1 GHz, 4GB Ram, SATA 7200rpm HDD, Nvidia GeForce 105M graphics card.</p>

<p>The spheres are initialized in random positions within a cubic box. The cubic box's walls can be (1) all &quot;transparent&quot; and without textures or (2) fully textured with the option of hiding any walls facing the camera.</p>

<h4>Collision, Deformation</h4>

<p>Spheres collide with each other and with the sides of the box. They change trajectories as expected with physics: the x component of the velocity is flipped, while the y component remains the same. Collision detection is optimized to only check first collision for each object, so it runs in O(n). On collision, sparks fly off in the plane perpendicular to the collision direction, whether the collision is between spheres or sphere-wall.</p>

<p>Deformation is done using NURBS to generate &quot;hills&quot; on the box.</p>

<h4>Light, Shadows</h4>

<p>We include one light source at the front of the box which casts real-time shadows over the (default 5) spheres. The light source moves in a similar trajectory to the objects. Notice that the spheres shadow each other as well, not just the walls of the box.</p>

<h4>Keyboard controls and more</h4>
<p>For usage details, please refer to the <strong>User's guide section</strong></p>

</div><!-- #tabs-2 -->
    <div id="tabs-3">
        <h3 class="hide">Goals and Timeline</h3>
        <a class="hide" href="#top">Back to top</a>
        <p><strong>Note:</strong><ul>
            <li>These goals are from our initial proposal and have not been updated. Please see our <strong>Checklist</strong> section for an updated list.</li>
            <li>We have successfully completed all our goals within the given timeline and added a few more features.</li><br/>
        </p>
        <table border="1">
            <tr>
                <th style="width: 15em;">Goal Name</th>
                <th style="width: 10em;">Asignee</th>
                <th style="width: 9em;">ETA</th>
                <th>Details <a href="#" style="display: none;">(Hide)</a></th>
            </tr>
            <tr>
                <td>Investigate randomization techiques given number of inputs</td>
                <td>Paul Craciunoiu</td>
                <td>Mon, Nov 16, '09</td>
                <td>Investigate how we decide to move objects around the screen based on input and random variables. There may be a need for noise or bias, depending on available input, and to ensure the visualizer offers satisfying visual output (i.e. is dynamic enough to look and feel in-tune with the audio).</td>
            </tr>
            <tr>
                <td>Experiment with available audio libraries and choose one for our project</td>
                <td>Derek Selander<br/>
                Paul Craciunoiu</td>
                <td>Wed, Nov 18, '09</td>
                <td>Choose an audio library which provides at least one line of numeric output based on input from audio file. Depending on read speed and CPU usage, we may use live data or do a two-pass to gather audio data first, and then use it it in our graphics engine. It would be better to have an audio library which provides more than one output per reading (e.g. low, high, mid frequency values), but this may also prove to be CPU intensive. A decision will be made in conjuction with an acceptable CPU usage decided during testing.</td>
            </tr>
            <tr>
                <td>Create 3-5 objects (assumed spheres) and place them in the viewport in perspective</td>
                <td>Derek Selander</td>
                <td>Fri, Nov 20, '09</td>
                <td>This likely includes implementing an object class, adding texture, color, shape, and creating the environment for the objects (maybe a room? maybe a number of clipping planes?)</td>
            </tr>
            <tr>
                <td>Create environment around objects</td>
                <td>Derek Selander</td>
                <td>Sun, Nov 22, '09</td>
                <td>Likely a room with walls which, at least initally, do not deform.</td>
            </tr>
            <tr>
                <td>Add object movement and collision (possible physics engine?)</td>
                <td>Paul Craciunoiu</td>
                <td>Wed, Nov 25, '09</td>
                <td>Ensure movement is smooth and feels like an animation. This step also allows testing of randomization and user input to observe and adjust behavior. Additionally, depending on the shapes of the objects, collision detection can range from easy to difficult. We are initially decided to choose spheres because of the ease in collision detection. Object trajectories may be subject to simulated physics rules for added complexity.</td>
            </tr>
            <tr>
                <td style="font-weight: bold; text-align: center;" colspan="3">PROJECT CHECKPOINT</td>
                <td></td>
            </tr>
            <tr>
                <td>Add camera movement</td>
                <td>Paul Craciunoiu</td>
                <td>Fri, Nov 27, '09</td>
                <td>Camera could rotate, zoom in and out, move, or affect perspective based on audio input.</td>
            </tr>
            <tr>
                <td>Add deformation when objects collide (with possible friction or energy loss?)</td>
                <td>Derek Selander</td>
                <td>Sun, Nov 29, '09</td>
                <td>This step, too, requires investigation of physics and simple vs realistic deformation on collision. Also, consideration for CPU usage.</td>
            </tr>
            <tr>
                <td>Add light sources along with existing objects</td>
                <td>Derek Selander</td>
                <td>Wed, Dec 2, '09</td>
                <td>Since light sources can only be finite in OpenGL, we expect to have at least 3. Initially, these light sources may be still, but if CPU usage allows, they may move. This step includes investigation into having multiple light sources and different types of light (diffuse, ambient, spotlight, etc) and roles that light sources can play in the visualizer. Possibly light sources can become moving objects themselves.</td>
            </tr>
            <tr>
                <td>Cast shadows of objects from light sources</td>
                <td>Paul Craciunoiu</td>
                <td>Fri, Dec 4, '09</td>
                <td>Light sources add shadows. This includes shading. Investigate different types of shadows and how realistic we can get, bounded by CPU usage.</td>
            </tr>
            <tr>
                <td>Lights situated closer to the viewer</td>
                <td>Derek Selander</td>
                <td>Sat, Dec  5, '09</td>
                <td>The lights would be closer to the viewer (higher z axis) and have a background set behind all the objects.  This would create shadows on the background.</td>
            </tr>
            <tr>
                <td>Finalize</td>
                <td>Paul Craciunoiu<br/>
                    Derek Selander</td>
                <td>Wed, Dec 9, '09</td>
                <td>Finishing touches</td>
            </tr>
            <tr>
                <td style="font-weight: bold; text-align: center;" colspan="3">WISHLIST</td>
                <td></td>
            </tr>
            <tr>
                <td>Objects can create other objects</td>
                <td>Paul Craciunoiu</td>
                <td>--</td>
                <td>We can add creation of other objects which are (or are not) independent of their parents. Creation of other objects may be based on sequence of received audio input with certain characteristics (e.g. high volume for 50% of last 5-10 reads turns some % of objects into "birth" mode)</td>
            </tr>
            <tr>
                <td>Objects can expire (or take damage)</td>
                <td>Derek Selander</td>
                <td>--</td>
                <td>While new objects arise, older objects may die out due to over-birth or by being hit with large or small numeric values.</td>
            </tr>
            <tr>
                <td>Objects can relate to each other (more than through collision)</td>
                <td>Paul Craciunoiu</td>
                <td>--</td>
                <td>For example, children may be destroyed when their parents die, or they may become rebellious and give birth n times in a short period of time (n &gt;1), or they may choose to "eat" or "adopt as parent" other objects</td>
            </tr>
            <tr>
                <td>Read in 3 or more numbers per reading from audio file (i.e. low, mid, and high frequencies).</td>
                <td>Derek Selander</td>
                <td>--</td>
                <td>Provides more information to generate better visual output</td>
            </tr>
            <tr>
                <td>Light sources can fluctuate (in intensity, color, other?)</td>
                <td>Derek Selander</td>
                <td>--</td>
                <td>Light sources could be influenced by the audio file as well and participate by changing the light intensity or color.</td>
            </tr>
            <tr>
                <td>Light sources can move</td>
                <td>Derek Selander</td>
                <td>--</td>
                <td>Moving light sources may add significant computational complexity.</td>
            </tr>
            <tr>
                <td>Objects can become light sources and/or vice-versa</td>
                <td>Paul Craciunoiu</td>
                <td>--</td>
                <td>Additional complexity by "spawning" light sources or having light sources die out (perhaps when their intensity is too low)</td>
            </tr>
            <tr>
                <td>Objects can become reflective</td>
                <td>Paul Craciunoiu</td>
                <td>--</td>
                <td>Object may reflect other objects (possibly one or more indirect reflections)</td>
            </tr>
            <tr>
                <td>Light sources deform</td>
                <td>Derek Selander</td>
                <td>--</td>
                <td>Light sources act like other objects and deform when they move around and collide.</td>
            </tr>
       </table>
    </div><!-- #tabs-3 -->
    <div id="tabs-4">
        <h3 class="hide">References</h3>
        <a class="hide" href="#top">Back to top</a>
        <ul>
            <li><a href="http://glprogramming.com/red/index.html">OpenGL Red Book</a></li>
            <li><a href="http://www.opengl.org/resources/code/samples/mjktips/Reflect.html">Fast reflections</a></li>
            <li><a href="http://magnum.dimajix.de/download/demos.shtml">OpenGL Demos</a></li>
            <li><a href="http://www.gnu.org/software/libcdio/doxygen/files.html">Audio library -- audio.h</a></li>
            <li><a href="http://www.portaudio.com/">Audio API -- PortAudio</a></li>
            <li><a href="http://www.portaudio.com/docs/v19-doxydocs/group__test__src.html">PortAudio examples</a></li>
            <li><a href="http://www.fltk.org/doc-1.1/glut.html">Porting GLUT to FLTK</a></li>
            <li><a href="http://www.gamedev.net/community/forums/topic.asp?topic_id=552890">Sliding reflection issues</a></li>
            <li><a href="http://www.xmms.org/docs/vis-plugin.html">OpenGL plugin for XMMS</a></li>
            <li><a href="http://www.opengl.org/resources/code/samples/mjktips/rts/index.html">Real Time Shadows</a></li>
            <li><a href="http://ozark.hendrix.edu/~burch/cs/490/sched/feb8/">Draw sphere code</a></li>
            <li><a href="http://www.videotutorialsrock.com/">Video Tutorials</a></li>
            <li><a href="#">OpenGL Super Bible, 4th edition</a>, Richard S. Wright, Jr</li>
            <li><a href="http://nehe.gamedev.net/">NeHe - GameDev - OpenGL lessons</a></li>
            <li><a href="http://www.swiftless.com/tutorials/opengl/basic_shadows.html">Basic Shadows</a></li>
            <li><a href="http://www.3ddrome.com/articles/shadowvolumes.php">Shadow Volumes</a></li>
            <li><a href="http://www.opengl.org/resources/code/samples/mjktips/TexShadowReflectLight.html">Shadow Example</a></li>
            <li><a href="http://icculus.org/~lucasw/OpenGL/OpenGL.html">OpenGL Spring model</a></li>
            <li><a href="http://www.paulsprojects.net/opengl/cloth/cloth.html">Cloth model</a></li>
        </ul>
    </div><!-- #tabs-4 -->
    <div id="tabs-5">
        <h3 class="hide">Media</h3>
        <p><a class="hide" href="#top">Back to top</a>
        This section contains screenshots, videos -- media from the project.
        </p>
        <p><img src="img/1.jpg" alt="image 1" width="321" height="286"> <img src="img/4.jpg" alt="image 4" width="310" height="286"></p>
        <p><img src="img/2.jpg" alt="image 2" width="319" height="289"> <img src="img/3.jpg" alt="image 3" width="312" height="290"></p>
        <p><a href="img/">See more images</a></p>
    </div><!-- #tabs-5 -->
    <div id="tabs-6">
        <h3 class="hide">Checklist</h3>
        <a class="hide" href="#top">Back to top</a>
        <p><strong>Note:</strong> This page shows our progress</p>
        <table border="1" width="100%">
            <tr>
                <th>Goal Name</th>
                <th style="width: 20em;">Comments</th>
            </tr>
            <tr>
                <td class="done">Investigate randomization techiques given number of inputs</td>
                <td>Using parametric curves</td>
            </tr>
            <tr>
                <td class="done">Experiment with available audio libraries and choose one for our project</td>
                <td>Using FMOD</td>
            </tr>
            <tr>
                <td class="done">Create 3-5 objects (assumed spheres) and place them in the viewport in perspective</td>
                <td></td>
            </tr>
            <tr>
                <td class="done">Create environment around objects</td>
                <td>Dark box without front wall</td>
            </tr>
            <tr>
                <td class="done">Add object movement and collision</td>
                <td></td>
            </tr>
            <tr>
                <td class="done">Add camera movement</td>
                <td>Roll, pitch and yaw <br/><strong>Yaw automatically</strong></td>
            </tr>
            <tr>
                <td class="done">Add light source close to edge of box</td>
                <td></td>
            </tr>
            <tr>
                <td class="done">Update movement equations to achieve independent movement for each axis</td>
                <td></td>
            </tr>
            <tr>
                <td class="done">Add collision with wall</td>
                <td></td>
            </tr>
            <tr>
                <td class="done">Add deformation when objects collide with wall</td>
                <td></td>
            </tr>
            <tr>
                <td class="done">Cast shadows of objects from light sources</td>
                <td>Real-time shadows implemented<br/>Our biggest performance hit</td>
            </tr>
            <tr>
                <td class="done">Finalize</td>
                <td></td>
            </tr>
            <tr>
                <td style="text-align: center;"><strong>WISHLIST</strong></td>
                <td></td>
            </tr>
            <tr>
                <td class="done">Add speed control</td>
                <td></td>
            </tr>
            <tr>
                <td class="done">Add sparks when objects collide</td>
                <td></td>
            </tr>
            <tr>
                <td class="done">Add texture mapping for box</td>
                <td></td>
            </tr>
            <tr>
                <td class="done">Read in 3 or more numbers per reading from audio file (i.e. low, mid, and high frequencies)</td>
                <td>Reading in 256x2 real values</td>
            </tr>
            <tr>
                <td class="done">Light sources can fluctuate (in intensity, color, other?)</td>
                <td>Light sources behave with music</td>
            </tr>
            <tr>
                <td class="done">Light sources can move</td>
                <td>Same as above</td>
            </tr>
            <tr>
                <td>Objects can create other objects</td>
                <td></td>
            </tr>
            <tr>
                <td>Objects can expire (or take damage)</td>
                <td></td>
            </tr>
            <tr>
                <td>Objects can relate to each other (more than through collision)</td>
                <td></td>
            </tr>
            <tr>
                <td>Objects can become light sources and/or vice-versa</td>
                <td></td>
            </tr>
            <tr>
                <td>Objects can become reflective</td>
                <td></td>
            </tr>
            <tr>
                <td>Light sources deform</td>
                <td></td>
            </tr>
       </table>
    </div><!-- #tabs-6 -->
    <div id="tabs-7">
        <h3>User's guide</h3>
        <a class="hide" href="#top">Back to top</a>
        <h4>Usage</h4>
        <code>./music-visualizer [-d(debug)|-h(help)|&lt;musicfile.[mp3|flac]&gt;]</code>
        <h4>Keyboard controls (case sensitive)</h4>
    <table id="controls" border="1" width="100%">
    <tr>
        <th>Task</th>
        <th>Key</th>
        <th>Key action</th>
        <th>Mouse button</th>
        <th>Mouse action</th>
    </tr>
    <tr>
        <th colspan="5">CAMERA MANIPULATION</th>
    </tr>
    <tr>
        <td>Zoom camera</td>
        <td>M</td>
        <td>press and hold</td>
        <td>middle</td>
        <td>drag</td>
    </tr>
    <tr>
        <td>Move camera horizontally or vertically</td>
        <td>H, V</td>
        <td>press and hold</td>
        <td>right</td>
        <td>drag</td>
    </tr>
    <tr>
        <td>Roll, pitch, yaw</td>
        <td>R, P, Y</td>
        <td>press and hold</td>
        <td>left</td>
        <td>drag</td>
    </tr>
    <tr>
        <td><strong>Reset</strong> camera</td>
        <td>r</td>
        <td>press</td>
        <td>&mdash;</td>
        <td>&mdash;</td>
    </tr>
    <tr>
        <th colspan="5">OTHER OPTIONS</th>
    </tr>
    <tr>
        <td><strong>Toggle</strong> perspective vs parallel projection</td>
        <td>p</td>
        <td>press</td>
        <td>&mdash;</td>
        <td>&mdash;</td>
    </tr>
    <tr>
        <td><strong>Toggle</strong> musical frequencies</td>
        <td>q</td>
        <td>press</td>
        <td>&mdash;</td>
        <td>&mdash;</td>
    </tr>
    <tr>
        <td><strong>Toggle</strong> trajectories of spheres</td>
        <td>f</td>
        <td>press</td>
        <td>&mdash;</td>
        <td>&mdash;</td>
    </tr>
    <tr>
        <td><strong>Toggle</strong> between showing target or direction of trajectory (must have above option enabled)</td>
        <td>d</td>
        <td>press</td>
        <td>&mdash;</td>
        <td>&mdash;</td>
    </tr>
    <tr>
        <td><strong>Toggle</strong> sparks on collision</td>
        <td>s</td>
        <td>press</td>
        <td>&mdash;</td>
        <td>&mdash;</td>
    </tr>
    <tr>
        <td><strong>Toggle</strong> auto-rotate (yaw)</td>
        <td>a</td>
        <td>press</td>
        <td>&mdash;</td>
        <td>&mdash;</td>
    </tr>
    <tr>
        <td><strong>Toggle</strong> pause/unpause</td>
        <td><code>&lt;space&gt;</code></td>
        <td>press</td>
        <td>&mdash;</td>
        <td>&mdash;</td>
    </tr>
    <tr>
        <td><strong>Toggle</strong> between texture and plain background.</td>
        <td>b</td>
        <td>press</td>
        <td>&mdash;</td>
        <td>&mdash;</td>
    </tr>
    <tr>
        <td><strong>Toggle</strong> background cube <strong>lines</strong></td>
        <td>B</td>
        <td>press</td>
        <td>&mdash;</td>
        <td>&mdash;</td>
    </tr>
    <tr>
        <td><strong>Toggle</strong> box outer faces</td>
        <td>1-6</td>
        <td>press</td>
        <td>&mdash;</td>
        <td>&mdash;</td>
    </tr>
    <tr>
        <td><strong>Change</strong> speed</td>
        <td>7-9</td>
        <td>press</td>
        <td>&mdash;</td>
        <td>&mdash;</td>
    </tr>
    <tr>
        <td><strong>Increase or decrease</strong> speed, respectively</td>
        <td>- or =</td>
        <td>press</td>
        <td>&mdash;</td>
        <td>&mdash;</td>
    </tr>
    <tr>
        <td><strong>Toggle</strong> origin axes</td>
        <td>w</td>
        <td>press</td>
        <td>&mdash;</td>
        <td>&mdash;</td>
    </tr>
    <tr>
        <td><strong>Toggle</strong> origin planes</td>
        <td>e</td>
        <td>press</td>
        <td>&mdash;</td>
        <td>&mdash;</td>
    </tr>
    </table>
    </div><!-- #tabs-7 -->
    <div id="tabs-8">
        <h3 class="hide">Readme</h3>
        <a class="hide" href="#top">Back to top</a>
        This section includes our readme file.
        <pre>
        </pre>
    </div><!-- #tabs-8 -->
</div>

</body>

<script src="http://ajax.googleapis.com/ajax/libs/jquery/1.3.2/jquery.min.js"></script>
<script src="http://ajax.googleapis.com/ajax/libs/jqueryui/1.7.2/jquery-ui.min.js"></script>
<script>

$(document).ready(function() {
    $("#tabs").tabs({
    });

    $('.hide').hide();
    $('th a').show().live('click', function () {
        $('table th:last-child').html('<a id="show" href="#">Show details</a>');
        $('table td:last-child').hide();
        $('table th:first-child').css('width', '');
    });
    $('#show').live('click', function() {
        $('table th:last-child').html('Details <a href="#">(Hide)</a>');
        $('table td:last-child').show();
        $('table th:first-child').css('width', '15em');
    });

    $.get("README", function(data) {
        $('#tabs-8 pre').html(data);
    });

    $('table td:nth-child(3)').css('text-align', 'center');
    $('table td:nth-child(2)').css('text-align', 'center');
    $('#controls td:nth-child(4)').css('text-align', 'center');
    $('#controls td:nth-child(5)').css('text-align', 'center');
});

</script>
</html>